TouchScale releases 500-hour vision-touch dataset, lifting robot task success from 22.5% to 57.5%
Researchers from 15 institutions, including Texas A&M University, Google DeepMind, Carnegie Mellon University, Stanford University, the University of California, Berkeley, Yale University, Microsoft, NVIDIA, Meta, and others, have introduced TouchScale, a 500-hour human vision-touch dataset collected with a single sensor stack and a unified capture and synchronization pipeline. The project is designed to test whether visual-tactile learning shows the same scaling gains seen in large video datasets when data quality and hardware consistency are held constant.
The dataset includes synchronized head-mounted RGB-D, wrist RGB video, and full-hand tactile signals from gloves with 880 taxels per hand. It spans about 87,000 interaction recordings, roughly 2,000 task descriptions, more than 1,500 objects, nine environment categories, over 800 scene configurations, and around 20 data collectors.
In experiments, TouchScale improved zero-shot cross-sensor tactile prediction and also produced the best visual encoder accuracy across MECCANO, Something-Something V2, and Ego-Exo4D when tactile signals were used as supervision. In real-world robot tests with an xArm6 and a BrainCo Revo 2 hand, adding TouchScale mid-training before post-training raised the average success rate across four contact-heavy tasks from 22.5% to 57.5%, without using human action labels or mapping human hand motions to robot actions.
